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ABSTRACT 

The literature on how educators use information to 
solve problems and perform their routine duties is located within a 
much larger, well-established literature that spans topics in 
organizational decision-making, communication theory, cognitive 
science, information theory, and evaluation utilization* This 
bibliography presents summaries of illustrative references for the 
general reader interested in information use in education. Most of 
the citations pertain directly to education, but some represent 
studies from related areas. Books, journal articles, major literature 
reviews, and reports of selected studies or experiments are included. 
The 43 citations include complete bibliographic information and a 
short abstract. They are numbered and arranged alphabetically, with 
an index at the end to provide access by topic. (Author/MAS) 
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The Center 



The mission of the Center for Research on Effective Schooling for Disadvantaged Students 
(CDS) is to significantly improve the education of disadvantaged students at each level of 
schooling through new knowledge and practices produced by thorough scientific study and 
evaluation. The Center conducts its research in four program areas: The Early and Elementary 
Education Program, The Middle Grades and High Schools Program, the Language Minority 
Program, and the School, Family, and Community Connections Program. 

The Early and Elementary Education Program 

This program is working to develop, evaluate, and disseminate instructional programs 
capable of bringing disadvantaged students to high levels of achievement, particularly in the 
fundamental areas of reading, writing, and mathematics. The goal is to expand the range of 
effective alternatives which schools may use under Chapter 1 and other compensatory education 
funding and to study issues of direct relevance to federal, state, and local policy on education of 
disadvantaged students. 

The Middle Grades and High Schools Program 

This program is conducting research syntheses, survey analyses, and field studies in middle 
and high schools. The three types of projects move from basic research to useful practice. 
Syntheses compile and analyze existing knowledge about effective education of disadvantaged 
students. Survey analyses identify and describe current programs, practices, and trends in middle 
and high schools, and allow studies of their effects. Field studies are conducted in collaboration 
with school staffs to develop and evaluate effective programs and practices. 

The Language Minority Program 

This program represents a collaborative effort. The University of California at Santa 
Barbara is focusing on the education of Mexican-American students in California and Texas; 
studies of dropout among children of recent immigrants are being conducted in San Diego and 
Miami by Johns Hopkins, and evaluations of learning strategies in schools serving Navajo, 
Cherokee, and Lumbee Indians are being conducted by the University of Northern Arizona. The 
goal of the program is to identify, develop, and evaluate effective programs for disadvantaged 
Hispanic, American Indian, Southeast Asian, and other language minority children. 

The School, Family, and Community Connections Program 

This program is focusing on the key connections between schools and families and between 
schools and communities to build better educational programs for disadvantaged children and 
youth. Initial work is seeking to provide a research base concerning the most effective ways for 
schools to interact with and assist parents of disadvantaged students and interact with the 
community to produce effective community involvement. 
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Abstract 



The literature on how educators use information to solve problems and perform their routine duties 
is less than twenty years old. However, it is located within a much larger, well-established literature that 
spans topics in organizational decision-making, communication theory, cognitive science, information 
theory, and evaluation utilization. 

This annotated bibliography is far from exhaustive. It is intended to present summaries of illustra- 
tive references for the general reader interested in information use in education. Major literature reviews arc 
included, as are reports of selected studies or experiments. Most of the citations pertain directly to educa- 
tion, but some representative studies from related areas are also included. 

The abstracts accompanying the bibliography entries were prepared either by expanding on an au- 
thor's own abstract or by extracting key sentences and phrases from an article or report. These abstracts 
thus are written primarily in the authors' own words. 

The citations in this bibliography are numbered and presented in alphabetical order. A brief index is 
provided at the end of the bibliography, so that citations pertaining to specific topics can be located quickly. 



Annotated Bibliography 



1. Algozzine, B., Ysseldyke, J. E., & 
Hill, C. (1982). Psychoeducational de- 
cision making as a function of the 
amount of information reviewed. 
Psychology in the Schools , 19. 328-334. 

Psychoeducational assessment is commonplace in 
America's public school systems, and important 
decisions for and about school children are made 
on the basis of assessment results, including test 
scores. In order to combat some of the problems 
inherent in test-based assessments, multi-disci- 
plinary team decision making has become current 
best practice. Parents, teachers, psychologists, 
and other professionals review assessment data as 
team members. However, individuals with little 
or no training in assessment are being asked to 
participate in decision making. This poses a 
problem for student assessment. 

This study was conducted to examine the effects 
of amount of information available to decision 
makers for review. Decision makers (elementary 
school teachers, school psychologists, adminis- 
trators, and other school staff) were allowed to 
review information from a variety of sources 
prior to making decisions about special education 
placement for a case study child. The researchers 
found that individuals who reviewed large 
amounts of information made decisions similar to 
those of individuals who reviewed little informa- 
tion. Approximately 49% of the participants 
made accurate decisions about placement when 
judged against an a priori criterion. These results 
suggest that increasing amounis of assessment in- 
formation bear little relationship to the psychoed - 
ucational decision being made. 

2. Bank, A. (1982). Can evaluation 
plus staff development equal school im- 
provement? The Journal of Staff 
Development. 3m. 170-181. 

Growing cadres of research- or evaluation-ori- 
ented persons in school districts, fortified with an 
expanding array of evaluative techniques, consti- 
tute a new force for data-based school improve- 
ment. However, many of these resource person- 
nel are isolated from teachers, principals, and 



even from other central office staff performing 
related functions, including staff development. 

The project reported in this article grew out of a 
survey from the Center for the Study of 
Evaluation on the use of evaluation and testing for 
school improvement. The author classifies and 
describes what six school districts did to relate 
evaluation to staff development. In the 
"evaluation for staff development" category, 
some districts used information derived from an 
analysis of test scores or other data to shape some 
or all of the district's staff development offerings. 
For example, one district found that many stu- 
dents were failing a high school proficiency exam 
because high school math teachers had not incor- 
porated checkbook balancing or comparative pric- 
ing into their algebra or geometry courses. Other 
districts used "evaluation as staff development," 
in which staff either received data from a central 
source or generated it themselves and used it to 
plan new objectives, strategies, and so on. For 
example, one central evaluation office assigned 
staff members to particular schools to work with 
them and help them obtain and analyze the data 
they needed. 

The author describes several problems with this 
approach. Many districts do not have adequate 
resources for using their research and evaluation 
offices in these ways; some districts do not regard 
this approach as having a high payoff; teachers 
arc felt to lack the training and skills to deal wilh 
data; teachers sometimes react negatively to data 
cither as invalid because it contradicts their per- 
sonal judgments or as superfluous because it 
corroborates their judgments or observations; and 
using evaluation results for local planning am 
easily become bureaucratized. 

The author concludes that data-based instructional 
change requires that district managers integrate 
separate instruction, evaluation, and staff devel- 
opment functions on the central office level and 
also develop coordinating mechanisms between 
the district and principals and teachers. Efforts in 
this direction seem to be of recent origin, with 
interest due to social and political forces calling 
for educational change. 
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3. Bank, A., & Williams, R. C. 
(1985). From program evaluation to 
instructional information system* 
Studies in Educational Evaluation. JLL, 
159-182. 

Instructional information systems (IIS) is the term 
given to computerized data banks maintained by 
an increasing number of schools and school dis- 
tricts. These information systems contain demo- 
graphic information about students, records of 
student test scores and grades, and other infor- 
mation about students, teachers, and schools. 
The computerized systems maintain this informa- 
tion in an integrated file that allows records to be 
sorted, combined, and analyzed in a variety of 
user-defined ways. By enabling a variety of 
users to ask important evaluative questions about 
student learning, classroom function, school 
management, and district policies, instructional 
information systems build on current practice in 
testing and program evaluation but go beyond it. 

The capability to analyze and report out informa - 
tion at many levels of specificity, on various time 
tables, and in report formats designed for the 
needs of any type of user will distinguish IIS 
from what might be termed computerized record 
keeping. These systems may have a significant 
impact on school organizational patterns and on 
the school as a workplace for adults and a learn- 
ing environment for students. For example, 
analyses may indicate the need for changing the 
sequence in which subject matter is introduced, or 
the need for restaffing or retraining. An instruc- 
tional infomiation system may transform job de- 
scriptions and organizational roles and responsi- 
bilities for teachers, supervisors, and administra- 
tors. 

As instructional information systems are devel- 
oped, the first concern is typically with technical 
problems: selection of hardware and software, 
determination of what information to store, and 
hiring and training of staff. Once these technical 
problems are resolved, administrative and inter- 
personal problems, as well as more fundamental 
organizational and ethical issues, will likely need 
to be addressed. 

The authors discuss a number of potential uses 
for instmctional information systems, factors in- 
fluencing the emergence of instructional informa- 
tion systems, the current status of testing, evalua- 
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tion, and instruction?! linkages in school districts, 
helpful guidelines from the management informa- 
tion systems literature, and emerging issues sur- 
rounding information systems in education. 

4. Bank, A., & Williams, R. C. (Eds.) 
(1987). Information systems and school 
improvement: Invent ing the future. New 
York: Teachers College Press. 

This book is about the use of computer-based 
information systems to support instructional man- 
agement decisions. It is based on the work of the 
Instructional Information Systems Project at the 
Center for the Study of Evaluation at UCLA. The 
chapters present conceptual, technical, adminis- 
trative, and organizational perspectives on in- 
structional information systems. Some of the 
topics covered include: reasons for the emergence 
of educational information systems; the connec- 
tions between information systems and educa- 
tional improvement; lessons from management in- 
formation systems; and how instructional infor- 
mation systems fit into the culture of the school or 
district. Several different types of instructional 
information systems being developed in districts 
and schools are described in detail, and technical 
aspects of instructional information systems arc 
discussed as well. 

5. Barnette, J. J., & Thompson, J. C, 
III. (1979). A descriptive assessment 
of the effect of evaluations on instne- 
tion. Studies in Education al Evaluation. 
£, 77-86. 

This article presents the results of a survey that 
examined teacher perception of: (1) the effects of 
evaluations on instructional improvement, and (2) 
tfie types of evaluation processes that seem to ef- 
fect change. The evaluation types under consid- 
eration were program evaluation, including 
school accreditation studies, teacher performance 
evaluation, and student performance evaluation. 
The study sample consisted of a random sample 
of 208 Instructional and Professional 
Development chairpersons in a northeastern state; 
all were full-time secondary classroom teachers 
elected by their local association of the state edu- 
cation association. 

Sixty-three percent of the respondents felt that 
program evaluation had not resulted in any 
change in their instructional procedures. 



Similarly, 72% of the respondents indicated that 
teacher evaluation had not resulted in iiny change 
in instruction. However, 93% of the respondents 
ir. ,: cated that student performance evaluation did 
result in a change of instruction. The type of stu- 
dent performance evaluation most frequently cited 
(49%) as resulting in instructional change was 
"observation of in-class work of students." 
"Conversations with students" and "teacher-made 
tests" were next most frequently cited (18% for 
each). 

The authors conclude that if evaluation processes 
are to remain an important component of educa- 
tion, then changes in program and teacher evalua- 
tion, and in how their results are disseminated to 
and used by teachers, should be made. 

6. Baron, J. (1988). Thinking and de - 
ciding . Cambridge, England: Cambridge 
University Press. 

This book is about the psychology of thinking 
and deciding. The author's basic approach is that 
thinking is a search-and-inference process 
undertaken in order to help a person achieve his 
or her goals. Descriptive models of thinking 
theories about how people normally think - are 
developed through methods that trace the process 
of thinking as it occurs, examine the conclusions 
that people draw or the decisions they make, or 
use computers to model the human thinking 
process. Normative models - theories about 
how people should think — arc developed by 
philosophical reflection. Prescriptive models -- 
theories designed to change thinking patterns -- 
include heuristics and decision rules and methods 
of logic or scientific method. 

Some topics covered in the discussion of thinking 
arc: rationality and belief, problem -solving 
strategies, basic learning processes, and intelli- 
gence and creativity. In the second section of the 
book, on probability and belief, the author dis- 
cusses accuracy of probability judgments, heuris- 
tics and biases in probability judgment, hypothe- 
sis testing, and judgment of correlation and con- 
tingency. The book's third section covers theo- 
ries of utility and choice, quantitative judgment 
and moral thinking, social dilemmas, and deci- 
sions about the future. The book concludes with 
a chapter on the teaching of thinking and decision 
making. 



7. Berkowitz, G., Hyman, I., & Lally, 
D. (1984, April). The development of a 
school wide computerized uniform dis- 
cipline reporting system. Paper pre- 
sented at the annual convention of the 
National Association of School 
Psychologists, Philadelphia, PA. (ERIC 
Document Reproduction Service No. ED 
253 970) 

Among the many proposed approaches to dealing 
with the publics concern for school discipline 
has been the development of models for improv - 
ing discipline reporting procedures, such as par- 
ent and student discipline handbooks and detailed 
codes which spell out infractions and their re- 
spective consequences. However, there has been 
no evidence to demonstrate the effectiveness of 
discipline codes in reducing problems. To ad- 
dress these issues, the staff at the National Center 
for the Study of Corporal Punishment and 
Alternatives in the Schools began a long term or- 
ganizational development project in Trenton, NJ. 
A preliminary survey of teachers and administra- 
tors in the district revealed the need for an accu- 
rate and easily obtainable data base in order to 
make discipline decisions. Teacher and Center 
representatives developed a codebook of offenses 
and discipline procedures. A coding sheet was 
keyed to the codebook, so that teachers could 
easily record infractions and disciplinary actions 
taken. Information from the coding sheets were 
entered into a microcomputer program, which 
could produce narrative descriptions of in:idcnts 
based on the codes entered, and could support 
daily printouts and various analyses of the infor- 
mation. This system has the potential to make a 
considerable amount of information available to 
school staff very quickly. 

8. Bickel, W. E. (1984). Evaluator in 
residence: New prospects for school 
district evaluation research. Educational 
Evaluation and Policy Analysis , fi(3), 
297-306. 

This article describes a collaboration between the 
Pittsburgh Public Schools and the Learning 
Research and Development Center at the 
University of Pittsburgh. The Center provided 
resident evaluators to document the design and 
implementation of two complex educational im- 
provement programs in the Pittsburgh schools: an 



elementary school improvement project, and a 
secondary school teacher center. The evaluators 
were able to serve six broad functions; (1) to help 
improve the program (through formal research 
activities to gather data for monitoring and 
reshaping program initiatives); (2) to serve as a 
technical resource; (3) to serve as a source of 
reference about work completed and outstanding 
issues; (4) to serve as a sounding board for ideas, 
conflicts, and frustrations; (5) to increase 
program cohesiveness by providing information; 
and (6) to improve a district's knowledge base. 
Factors that influenced the utility of the resident 
evaluator's role included the researcher's 
integration into the program, the client's 
orientation toward research and evaluation, the 
strength of the focus on program improvement, 
and the researcher's ability to be methodologically 
eclectic. 

9. Bickel, W. E., & Cooley, W. W. 
(1985). Decision-oriented educational 
research in school districts: The role of 
dissemination processes. Studies in 
Educational Evaluation.il. 183-203. 

This article describes aspects of the collaboration 
between the Learning Research and Development 
Center at the University of Pittsburgh and 
Pittsburgh Public Schools. When the collabora- 
tion began, the school district did not have its 
own research office. The work undertaken 
through the collaboration included studies of 
program impact and needs assessments to estab- 
lish district or program priorities. In the process, 
the researchers shifted from a focus on 
"evaluation research" to "decision-oriented edu- 
cational research" (DOER) for three reasons: it 
diminishes the restrictions on research methods 
usually associated with evaluation research, it 
does not carry the negative baggage the term 
evaluation has, and it highlights the goal of being 
useful for decision making. 

Three strategies characterize decision-oriented ed- 
ucational research. The first is the use of a con - 
tinuous process of data collection and analysis, 
called monitoring and tailoring. The second is the 
adoption of a client orientation as a way of orga - 
nizing research. The third is the documentation 
of program implementation, in order to enable 
decision makers to more systematically lcam from 
past efforts. 



The authors review some current thinking about 
educational research utilization and summarize 
factors that influence use. They then describe the 
process of disseminating results of two research 
projects. The first was a district-wide needs as- 
sessment. Dissemination activities included fre - 
quent, informal updates by the researchers for the 
superintendent, beginning about a month after the 
research was begun; a preliminary presentation of 
formal results to district administrators to guide 
revisions to the presentation format; a formal 
presentation to the Board of Education in an all- 
day retreat; and a series of meetings with various 
components of the district's staff. In the second 
program, which was a documentation of a school 
improvement program, dissemination involved 
weekly meetings with the school planning team to 
share insights, along with a more formal 
dissemination of the results of a survey. 

The researchers came to believe that verbal com- 
munication of research results is at least as valu- 
able, and in some instances more valuable, than 
the written word. They also Pressed the impor- 
tance of tailoring the content of reports - whether 
verbal or written — to the information needs of 
specific audiences. 

10. Bluhm, H. P. (1987). 
Administrative uses of computers in the 
schools . Englewood Cliffs, NJ: 
Prentice-Hall, Inc. 

Schools have become more interested in com- 
puter-based information processing due to in- 
creased demands for information by federal and 
state agencies and accrediting bodies, growth in 
size and scope of school system activities, and the 
speed, accuracy, and storage abilities of computer 
systems. School business administrators were 
among the first to recognize how the computer 
could assist them in accounting for funds, pro- 
cessing payrolls, and maintaining personnel 
records. Computer technology, however, has 
implications for all aspects of the educational 
system, including facilities, equipment and sup- 
plies, instructional programming, and student 
records. Superintendents and building principals 
have come to recognize that computers are ideally 
suited to process information that can be used in 
making management decisions. This book dis- 
cusses technical and organizational considerations 
in developing computer information systems, and 
then discusses applications for managing the ad- 



ministrativc office, managing business opera- 
tions, managing the educational program 
(scheduling, attendance, test scoring, etc.), and 
managing the instructional program (computer- 
managed instruction). Several chapters describe 
considerations in selecting various types of edu - 
cational software. 

11. Blum, R. E., & Butler, J* A. 
(1985). Managing improvement by 
profiling. Educational Leadership. 
42(6), 54-58. 

In this article, the authors describe a process 
called "profiling" that is used to gather informa- 
tion about student performance in a school. The 
resulting school profile can then be used as the 
basis for planning and implementing school im- 
provements. Profiling is part of an overall school 
improvement program sponsored by the 
Northwest Regional Educational Laboratory. 
School leadership teams plan for profiling, intro- 
duce the process to staff, collect and summarize 
schoolwide data, and prepare a written report, or 
profile, of the school. Copies of the profile are 
distributed to all staff members for consideration 
before making improvement goal decisions. Staff 
members are asked to respond to the profile in 
terms of their relative satisfaction with the current 
picture of student performance, the relative 
importance of improvement among areas with 
which staff members are dissatisfied, the 
acceptable standard of performance in areas of 
least .satisfaction, and a specific target for 
improvement. Using this feedback, the 
leadership team develops improvement goals for 
the school. The school profile thus provides 
baseline data about student performance against 
which effects of school improvements can be 
measured. This information becomes a primary 
tool in planning and managing targeted school 
improvement efforts. 

12. Borgida, E., & Nisbett, R. E. 
(1977). The differential impact of ab- 
stract vs. concrete information on deci- 
sions. Journal of Applied Social 
Psychology. 7m. 258-271. 

Recent findings from research on judgment and 
attribution processes indicate that people regard 
base rate data, i.e., statistical summaries of popu- 
lations, as if they were uninformative. It is sug- 



gested that base rate information lacks impact be- 
cause of its abstract, pallid nature. In this exper- 
imental study, college undergraduates were given 
written course evaluations based on the average 
rating scores of students who previously took the 
courses, and were asked to list the courses they 
would likely enroll in. The base rate information 
had little impact on course choices. In contrast, 
when students heard brief comments about the 
courses presented by other students in face-to- 
face encounters, this information had a substantial 
impact on course choices. The results suggest 
that information is utilized in proportion to its 
vividness. 

13. Burstein, L. (1984). The use of 
existing data bases in program evaluation 
and school improvement. Educational 
Evaluation and Policy Analysis. £(3), 
307-318. 

The purpose of this article is to comment on prac- 
tice in the use of existing data bases in program 
evaluation and school improvement and to ex- 
plore directions of increased and improved use. 
To date, the record of local district practice in 
maintaining and using data archives is limited; at 
present, local uses of data tend to be responsive 
and targeted rather than reflective and multi- 
faceted. The author assumes that the current state 
of affairs in information maintenance and use in 
local districts is more a happenstance of compet- 
ing priorities for economic and human resources 
and limited technical expertise and support than a 
conscious judgment that current practices exhaust 
the benefits that might accrue. 

The author considers the debate on test score de- 
clines in 1970s as an example of how local dis- 
tricts' information maintenance and analysis prac- 
tices made them unable to contribute to the dia- 
logue or to document whether national trends evi- 
denced in various policy reports applied to their 
local situation. Although local school districts are 
a long way from making full use of information 
from their recurring data collection activities, it is 
reasonable to anticipate growth in multiple uses of 
existing data. Some of the functions that existing 
data in local schools can serve, if maintained in 
accessible and documentable form, include long 
range planning; pulse monitoring; student deci- 
sion making; program decision making; and in- 
forming educational policy. 
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These functions fall within the realm of possibil - 
ity, if sufficient economic resources and interest 
are present. Local districts need the following: 
(1) a general commitment from participants in the 
educational community to the value of using 
existing data for informed inquiry and educational 
change; (2) sufficient computer and data literacy 
on the part of students, teachers, administrators, 
and parents to benefit from the availability of data; 
(3) computer hardware and software capable of 
providing ready and timely access to information, 
along with the ability to transmit information to 
agencies external to districts for broader policy 
analysis efforts; (4) sufficient computer and data 
management and analysis expertise to create, 
maintain, and analyze data archives; (5) sufficient 
safeguards to ensure protection of personal pri- 
vacy; and (6) a healthy and informed understand- 
ing of the limits as well as the possibilities of in - 
formation-based decision making. 

14. Clark, C. ML, & Yinger, Robert J, 
(1977)* Research on teacher thinking. 
Curriculum Tnqmrv. 7 (4). 279-304. 

This article reviews research about teachers' be- 
liefs and their judgment, decision making, and 
planning. It is based on the approach to the study 
of teaching that assumes that what teachers do is 
affected by what they think. 

The authors first review research evidence about 
teacher planning. The available literature sug- 
g sts that teachers do not seem to follow the 
"rational moder that is often prescribed in teacher 
training and in curriculum planning. The teachers 
studied did not begin or guide their planning in 
relation to clearly specified objectives or goals; 
rather, teacher planning seems to begin with the 
content to be taught and considerations about the 
setting in which the teaching will take place. The 
activity, rather than the objective, seems to be the 
unit of planning; student involvement is a sec - 
ondary consideration. Then, the authors consider 
teacher judgment. The small number of studies 
of teacher judgment yield less clear cut results. 
The evidence is mixed on the extent to which 
teachers' judgments are flexible and responsive to 
new information. It is clear that teachers vary in 
the accuracy of their predictions of student 
achievement and the weights that they assign to 
factors that influence their judgment. 



The studies of teacher interactive decision making 
(decision making during teaching) show that in - 
teractive decision making occurs primarily at 
times when there are interruptions of the ongoing 
instructional process by students. When inter- 
ruptions occur (often due to changes in student 
level of involvement), teachers occasionally con- 
sider alternatives but hardly ever implement them. 

Finally, the authors consider the idea that teach- 
ers' implicit theories about teaching may have an 
effect on their judgment and decision making. 
Some research has suggested that this connection 
between a teacher's implicit theory and teacher 
behavior is a relatively loose one, mediated by 
circumstances such as availability of resources, 
peer influence, and student characteristics. 

15. Clark, C. M., & Peterson, P. L. 
(1986). Teachers' thought processes. 
In M. C. Wittrock (Ed.), Handbook of 
research on teaching (3rd ed.) (pp. 255- 
296). New York: Macmillan Publishing 
Company. 

This chapter reviews research on teacher think- 
ing. It briefly traces the early history of research 
in this area, beginning with a 1968 book that tried 
to describe the mental constructs and processes 
that underlie teacher behavior. After presenting a 
heuristic model of the relationship between teach- 
ers' thought processes, their actions, and observ- 
able effects on students, followed by a discussion 
of the primary methods of inquiry into teacher 
thinking, the authors review research on teacher 
planning, teacher interactive decision making, and 
teacher beliefs. 

In the section on planning, the authors discuss the 
relationship between different types of teacher 
plans. For example, written lesson plans are 
nested within more comprehensive planning 
structures or "images" of teaching, and the 
"problem space" defined by plans made early in 
the school year exerts a powerful influence 
later plans. Plans often result in the development 
or selection of routines - instructional, manage- 
ment, or activity routines - that reduce but do not 
eliminate the unpredictability and uncertainty of 
the classroom for the teacher. Next, the authors 
review research on interactive decisions, focusing 
on the frequency of interactive decisions, teach- 



crs' consideration of alternative courses of action, 
antecedents of interactive decisions, and their re- 
lationship to teacher effectiveness. Finally, the 
authors discuss teachers' theories and beliefs 
about teaching and students, including teacher at- 
tributions, implicit theories about teaching, and 
teachers 1 perspectives of their role. 

16. Clemson, B. (1980). Harnessing 
the computer in educational management. 
Journal o f Educational Administration. 
i£(l), 98-113. 

Computer applications in education have generally 
either (1) streamlined some existing routine 
operation such as payroll or scheduling or (2) at- 
tempted some very sophisticated modeling of, for 
example, learning processes or cost functions for 
an inst : tution. None of these applications has 
made a noticeable impact on the practices or the 
management of education. The proper function 
of the computer, claims the author, is as a filter to 
suppress irrelevant data and highlight critical data, 
to make short term projections of trends, to vi- 
sually display the dynamics of system intcrac - 
tions, and to allow managers and teachers to in- 
teract with models of the system (e.g., the student 
or the school) they arc supposed to be in charge 
of. This paper sketches the outlines of a com- 
puterized information system designed to use the 
strengths of the computer to complement the 
strengths of the human teacher and manager. 

The typical information system for a school, 
whether computerized or manual, includes a great 
deal of data on students. But this data on indi- 
viduals is spread out in time and space, and it is 
impossible to gather together the relevant details 
to find out what is happening to the individual. 
The educational system collects and uses data on 
classes, on group test scores, on race and income 
levels, on cafeterias and buildings, on student 
flow through various curriculae, and so on. The 
teacher is the only point at which data on students 
as individuals is used. These observations lead 
the author to conclude that the educational system 
is managing classes, age cohorts, bus schedules, 
and so on, while the teacher alone attempts to 
manage students and their learning. The individ- 
ual must be the unit around which data is orga- 
nized. What is needed is a single file which re- 
ceives all of the important data on individual stu- 



dents. This data base can provide all of the in- 
formation currently produced about students. It 
can permit machine monitoring and exception re- 
porting on each student; in other words, each stu- 
dent's progress in all subjects can be automati- 
cally monitored. The student-based system might 
well provide for vastly improved decisions about 
each student. 

17. Cooley, W. W., & Bickel, W. E. 
(1986). Decision-oriented educational 
research . Boston: Kluwer-Nijoff 
Publishing. 

This book describes in detail a joint project of the 
Learning Research and Development Center at the 
University of Pittsburgh and the Pittsburgh 
Public Schools. For several years, university 
personnel functioned as researchers for the public 
school superintendent and board of education, 
contributing to the policy making process regard- 
ing a number of major issues. Through this col- 
laboration, the authors developed an approach to 
educational research that focuses on providing in- 
formation that can contribute directly to educa- 
tional decisions. 

The first chapters of this book present the frame- 
work for decision-oriented educational research 
and discuss such key components as the impor- 
tance of taking a client perspective, the need for 
an ongoing process of data collection and analy- 
sis, the nature and importance of computer-based 
information systems that can permit researchers to 
respond to client information needs in a timely 
fashion, the ^le of student achievement data in 
analyzing system performance, and the role of 
flexible dissemination processes in increasing the 
utilization of research results. Then, the authors 
present and discuss six case histories of decision- 
oriented research. The case histories include ex- 
amples of traditional program evaluations (e.g., 
evaluations of a Title I program and a new read- 
ing program); needs assessments (both district- 
wide ana at the middle school level); documenta- 
tion of complex innovations; a study of the use of 
achievement data for analyzing administrator 
performance; and an analysis of the effects of 
grade reorganization on student achievement. 
The cases are u^cd to illustrate the strategics and 
methods, as well as some of the problems, of 
decision-oriented educational research. 
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18, Cox, G, B. (1977), Managerial 
style: Implications for the utilization of 
program evaluation information. 
Evaluation Quarterly. If 3). 499-508. 

This article describes six characteristics of man - 
agerial work activity and draws implications for 
increasing the utilization of program evaluation 
data. The six characteristics are: (1) a manager's 
job requires much work at an unrelenting pace; 
(2) work activity is characterized by brevity, va- 
riety, and fragmentation; (3) managers prefer ac- 
tion; (4) managers prefer verbal communication; 
(5) the manager is active as a communication link 
with those above, below, and outside the organi- 
zation; and (6) managerial work is characterized 
by a blend of rights and duties - sometimes in 
control, sometimes operating under constraints. 

This model of managerial behavior has at least 
four areas of application for evaluation in human 
services programs. First, evaluation data will 
probably never be the sole source of information 
about a program, or the most important from the 
manager's point of view. Therefore, the relevant 
decision is not likely to follow an evaluation re- 
port as quickly as an e valuator might wish, but on 
the other hand, data may persist in the system and 
be used at a much later time. Second, evaluation 
results will be utilized to the degree and in the 
way in which they are relevant to an issue of in - 
terest to the manager, and these are usually con- 
crete and specific. Therefore, evaluation results 
should be oriented toward specific questions 
which are of interest to program managers. 
Third, managers are likely to have a rather casual 
attitude toward issues of validity. And fourth, 
although written reports are usually necessary, 
the real communication, insofar as the manager is 
concerned, will be verbal. 

19« Dorr-Bremme, D. W. (1983), 
Assessing students: Teachers 1 routine 
practices and reasoning. Evaluation 

Comment, 6.(4), 1-12. 

This article reports on the Center for the Study of 
Evaluation's Test Use in Schools Project. The 
author first presents an overview of the project's 
research activities from 1979 through 1982. The 
project utilized a broad definition of test and test- 
ing, to encompass a wide range of types of formal 
assessment measures, including commercially 
produced norm- and criterion-referenced 



measures; tests of minimal competency or func- 
tional literacy; and district-, school-, and teacher- 
constructed tests. Less formal measures for 
gauging student achievement, such as teachers' 
observations of and interactions with learners, 
were included as well. 

The inquiry focused on achievement assessment 
practices and uses in reading/English and 
mathematics as carried out in public schools at the 
upper-elementary and high school levels. 
Research activities included a literature review, 
exploratory interviews with staff from schools 
around the country, a nationwide survey of 
teachers and principals, and a study of testing 
costs. This article synthesizes and interprets 
findings from these research activities. 

The findings of the Test Use Project suggest that 
teachers think and act both as practical researchers 
and decision makers and as clinicians. They ori- 
ent their assessment activities to the practical tasks 
they have to accomplish in their everyday routines 
- determining what to teach and how to teach it, 
determining whether what they teach is being 
learned and adjusting instruction as necessary, 
and giving students grades. Moreover, teachers 
take into account all of the data at hand and make 
sense of students' academic performances clini- 
cally. It does not seem as if teachers base their 
decisions primarily on one kind of assessment in- 
formation, then look to others merely for confir- 
mation. Rather, they appear to weigh various 
kinds of data on student achievement collectively 
and to make sense of what it means more-or-less 
holisticaUy - a practice typical of clinical profes- 
sions. 

The author argues that if testing programs are to 
be useful for teachers and used in classrooms, 
they must take into account teachers' routine 
thinking and practices in assessing students' 
achievement. The Center's research results indi- 
cate that teachers favor tests that are proximal to 
the everyday instructional tasks teachers need to 
accomplish, consonant with the curriculum that 
teachers are actually teaching, immediately acces- 
sible, and designed to include different types of 
response formats and tasks. 

The article concludes with a description of test 
use in one illustrative scho; i district. 
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20. Eckholm, D. J. (Summer, 1983). 
Characteristics of a comprehensive stu- 
dent records system. Cflilggfi and 
UniYfirsitY,lS(4), 376-383. 

This article describes the broad, general attributes 
that would characterize a comprehensive student 
records system. Such a system should be: (1) 
integrated (information maintained in a central file 
rather than in multiple files) in order to reduce 
data redundancy and increase reliability; (2) cu- 
mulative (one file per student, not a separate file 
for each term that a student was enrolled); (3) his- 
torical (with information saved for a long time, 
first on an active file, then an inactive file, and 
then perha;.s stored on archived tape); (4) re- 
sponsive (the system should provide timely ser- 
vice to requests for information, and should be 
able to respond to ad hoc requests for information 
and be able to adapt to changes in data require- 
ments; (5) flexible (allowing data elements to be 
added or deleted, code changes to be made, and 
new technology to be incorporated; and (6) utiliz- 
ing efficient and cost-effective methods of data 
entry and retrieval. 

21. Einhorn, H. J., & Hogarth, R. M. 
(1978). Confidence in judgment: 
Persistence of the illusion of validity. 
Psychological Review . 8 5(5^ 395-416. 

An accumulating body of research on clinical 
judgment, decision making, and probability esti- 
mation has documented a substantial lack of 
judgmental ability for both experts and nonex- 
perts. However, evidence shows that people 
have great confidence in their fallible judgment. 
This article examines why the illusion of validity 
persists and how this contradition can be re- 
solved, and in so doing, discusses the relation- 
ship between learning and experience. 

In real-world situations, judgments are made for 
the purpose of choosing between actions. This 
means that outcome information, which is avail- 
able only after actions are taken, is frequently the 
only source of feedback with which to compare 
or evaluate judgment. But people often receive 
feedback primarily on the "good" judgments they 
have made, and they tend to remember this in- 
formation better, they obtain less, and make less 
use of, information that disconfirms their judg- 
ments. Moreover, outcomes are coded in mem- 
ory as frequencies rather than probabilities, which 



has major implications for explaining the persis- 
tence of the illusion of validity. After reviewing 
research on the structure of judgmental tasks, the 
extent to which people can observe the outcomes 
of judgments, and how outcomes are coded and 
interpreted, the authors briefly discuss ap- 
proaches to improve one's ability to learn from 
experience and make better judgments. 

22. Feldman, M. S., & March, J. G. 
(1981). Information in organizations as 
signal and symbol. A dministrate 
Science Quarterly . 2 6. 171-186. 

Formal theories of rational choice suggest that 
information about the possible consequences of 
alternative actions will be sought and used only if 
the precision, relevance, and reliability of the in- 
formation are compatible with its cost. However, 
empirical studies of information in organizations 
portray a pattern that is hard to rationalize in such 
temis. In particular, organizations systematically 
gather more information than they use, yet con - 
tinue to ask for more. The authors suggest that 
this behavior is a consequence of some ways in 
which organizational settings for information use 
differ from those anticipated in a simple decision- 
theory vision. In particular, the use of informa- 
tion is embedded in social norms that make it 
highly symbolic. The gathering of information 
provides a ritualistic assurance that appropriate 
attitudes about decision making exist; it is a repre- 
sentation of competence and a reaffirmation of 
social virtue. Some of the implications of such a 
pattern of information use are discussed. 

23. Fuchs, L. S., Deno, S. L., & 
Mirkin, P. K. (1984). The effects of 
frequent curriculum-based measurement 
and evaluation on pedagogy, student 
achievement, and student awareness of 
learning. American Educational 
Research TonrnaL 21(2^ 449-460. 

When teachers integrate more frequent assess- 
ments of student achievement into their instruc - 
tional programs, they may be able to structure 
teaching better and provide more feedback to help 
students recognize effective learning strategics. 
In special education, teachers are required by law 
to assess student progress as part of their individ- 
ualized educational programs (IEPs). However, 
teachers appear to prefer unsystematic observa- 
tions to objective measurement. This article ex - 
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amines the effects of a repeated, systematic as- 
sessment system on student learning. 

Special education reading teachers used standard- 
ized tests to measure reading progress at frequent 
intervals (at least twice weekly) and made a mod- 
ification in the instructional program when, after 
seven to ten measurement points, it appeared that 
student progress was insufficient to attain speci - 
fied long-term reading goals. Children whose 
teacheis employed the ongoing measurement and 
evaluation system showed greater achievement on 
a specific reading task and more general achieve- 
ment measures than did students whose teachers 
used conventional monitoring methods, such as 
periodic teacher-made tests, informal observa- 
tions, and workbook samples. In addition, the 
experimental teachers appeared to be mors realis- 
tic about and responsive ro student progress. 
Finally, students were more knowledgeable about 
their own learning as a result of the systematic 
measurement and evaluation treatment. This find- 
ing supports the hypothesis that student knowl- 
edge of goals may affect school performance. 

(Note: For a related article, see Fuchs, L. S., 
Fuchs, D., Hamlett, C. L., & Hasselbring, T. S. 
(1987). Using computers with curriculum-based 
monitoring: Effects on teacher efficiency and sat- 
isfaction. Journal of S pecial Education 
Teclinology. £(4), 231-236.) 

24. Fuchs, L. 5*, & Fuchs, D. (1986). 
Effects of systematic formative evalua- 
tion: A meta-analysis. E&£kciimi3l 
£tuldE£a ? Li(3) ? 199-208. 

The ATI (Aptitude Treatment Interaction) ap- 
proach to learning presumes that specific learner 
characteristics, or aptitudes, interact predictably 
with certain types of instructional programs, or 
treatments, to produce comparatively strong stu- 
dent learning. However, the ATI approach to 
individualized special education has produced 
disappointing results, largely because it is difficult 
to specify learner characteristics adequately and 
not enough is known about the interactions 
among learner and teacher characteristics, educa- 
tional treatments, and classroom environments. 
A more inductive approach to individualized in- 
struction is systematic formative evaluation, in 
which there is ongoing evaluation and modifica- 
tion of student programs. Specifically, this ap- 
proach employs regular monitoring of handi- 



capped students* performance under different in- 
structional procedures, in order to provide a data 
base with which individualized programs may be 
developed empirically. Student programs arc 
modified when student progress is lower than 
desired. 

This meta-analysis of 29 studies found that stu - 
dent achievement was higher under systematic 
formative evaluation conditions than under control 
conditions. The findings suggest that one can 
expect handicapped students whose individual- 
ized educational programs are monitored sys- 
tematically and developed formatively over time 
to achieve, on average, .7 standard deviation 
units higher than students whose programs are 
not systematically monitored and developed for- 
matively. 

A less predictable finding of the meta-analysis 
was the significant difference associated with 
data-evaluation methods. When teachers were 
required to employ data-utilization rules, effect 
sizes were higher than when data were evaluated 
by teacher judgment. Data-evaluation rules re- 
quired practitioners to analyze student perfor- 
mance at regular intervals and, if the data sug- 
gested certain patterns, to introduce instructional 
changes into a student's educational program. 
The method by which data were displayed also 
produced a significant finding. When data were 
graphed, effect sizes were higher than when data 
simply were recorded. There are at least two 
possible explanations for this finding. First, a 
graphed data display may allow teachers to ana- 
lyze student performance trends more accurately 
and frequently; second, graphs may facilitate 
more frequent performance feedback directly to 
pupils. 

Although some special education practitioners 
may object to systematic formative evaluation be- 
cause of its time-consuming nature, the magni- 
tude of effects suggests that systematic formative 
evaluation may be worth additional teacher time. 
Moreover, recent developments in computer 
software, which facilitate the collection, storage, 
graphing, and analysis of student performance 
data, indicate that technology may improve the 
feasibility of implementing this methodology for 
specifying effective individualized instructional 
programs. 



16 



10 



25. Glasman, N. S. (1985). 
Perceptions of school principals about 
their engagement in evaluation on the 
basis of student data. Studies in 
Educational Evaluation. LL 231-236. 

This article describes and examines selected data 
on school principals' reported attitudes and be- 
havior associated with their engagement in evalu- 
ation processes that are based on student 
achievement data. Subjects for the study were 
271 elementary school principals in California. 
Five key concepts were developed to define prin- 
cipals' perceptions about student ; hievement uti- 
lization in evaluation as a managerial tool. These 
concepts were commitment to influencing student 
achievement, control over data utilization, use of 
data to evaluate programs and teachers, effective- 
ness and accountability in data utilization, and 
principals' own accountability in data utilization. 
Results showed the presence of strong beliefs and 
self-reported behavior associated with utilizing 
student achievement data in evaluation and man- 
agement. 

26. Herman, J. (1987), Evaluation for 
school improvement: trv-nnt of a com- 
prehensive school based model . Los 
Angeles: Center for the Study of 
Evaluation, UCLA. (ERIC Document 
Reproduction Service No. ED 285 894) 

This article reports on the Multilevel Evaluation 
Systems Project, a project sponsored by the 
UCLA Center for the Study of Evaluation, which 
sought to develop and implement a "top-down, 
bottom-up" evaluation model in several school 
districts. The model starts with the assumption 
that evaluation can be a valuable tool for improv- 
ing schools, but that the actual impact of evalua- 
tion information has been quite modest. Some 
possible explanations are that much evaluation 
practice has focused in a limited way on standard - 
ized tests; teachers and administrators are seen as 
data providers rathtr than data users; and the 
methods and paperwork requirements of top- 
down evaluation approaches have intruded into, 
rather than supported, teachers' own planning 
and improvement efforts. 



Because actual control over instruction and in- 
structional improvement resides in the school 
building, and also because schools and 
classrooms encompass tremendous diversity in 
student population, teacher skills, curricular 
goals, and teaching strategies, the appropriate unit 
for solving many educational problems is the 
school. But since individual schools may lack 
sufficient resources, expertise, or control to solve 
all of their problems, solutions often require 
direction, resources, and actions at higher 
administrative or governance levels. These 
realities suggest the desirability of a distributed 
system of evaluation which could provide local 
schools with a rich information base to aid their 
problem solving but which could also provide 
appropriate aggregate information for decision 
making at high levels of the system. 

The researchers developed a prototype model for 
field testing in five school districts. Researchers 
and district staff worked together to decide what 
needs the evaluation system should serve, what 
data should be included in the core data base, and 
how data should be collected, analyzed, and re - 
ported. The planning process identified a set of 
questions which school personnel hoped the 
evaluation system would help them answer, these 
questions concerned the outcomes of schooling 
for students, the nature and effectiveness of the 
educational process, and the influence of the 
context in which instruction occurs (including 
school climate and student background). From 
these questions, a set of measurable indicators 
was developed, along with a set of desired analy- 
ses and reports. 

In general, district superintendents were more in- 
terested than teachers in looking at trends over 
time and were more sophisticated in their desire to 
analyze the data in depth. Teachers, in keeping 
with their responsibilities, were more satisfied 
with simple bar charts which enabled them to ana- 
lyze their classes at single points in time. Overall, 
the implementation of the multi-level evaluation 
system required sustained attention to the organi- 
zational and socio-political factors which facilitate 
change in schools, and school personnel needed 
considerable support in envisioning a system and 
how it might be used to help them accomplish 
their responsibilities. 




27. Kennedy, M. M. (1984). How evi- 
dence alters understanding and deci- 
sions. Educational Evaluation and 
Policy Analysis , &(3), 207-226, 

This article explores the ways in which educators 
think about and use various kinds of "evidence" - 
- information from program evaluations, social 
experiments, management information systems, 
and other sources. Data for the study were gath- 
ered from interviews and observations of group 
meetings in 16 school districts; participants in- 
cluded superintendents and school board mem- 
bers, program managers, principals, and teach- 
ers. The researcher analyzed occurrences in the 
interviews and meetings when participants re- 
ferred to the use of data. 

The author categorized the ways in which study 
participants reported or demonstrated the use of 
data. Twenty-nine percent of the citations were 
for conceptual use of information (acquiring fac- 
tual knowledge and drawing inferences or con- 
clusions). Another 39% of the citations referred 
to information use in group interactions, for in- 
forming, persuading, or supervising others, or 
for responding to others. Thirty-two percent of 
the citations were for direct applications of infor- 
mation: complying with regulations, sorting and 
placing children, and other miscellaneous appli- 
cations. These uses were often interrelated, with 
one use leading to another. 

Some citations of infomiation use illustrated the 
ways in which already-available elements of 
working knowledge can be combined with new 
evidence in order to interpret the evidence. By 
drawing on elements of working knowledge, 
participants are able to connect the evidence to a 
larger picture of their working situations, thus 
extending its meaning, its relevance, and its ap- 
plicability. 

The districts participating in the study engaged in 
a considerable volume of interactive problem 
solving. In order to examine the ways in which 
evidence was used in group interaction to create a 
body of shared knowledge, the author analyzed a 
series of episodes of information use. In several 
episodes, participants were able to arrive at 
agreed-upon interpretations, while in others they 
were not. Similarly, group interaction resulted in 
a decision being made in some cases, a delayed 
decision in others, and no excision in still others. 



The author describes some conditions that ap- 
peared to characterize each pattern of information 
use. 

Two models of information use have been ad - 
vanced in the literature: an instrumental model, in 
which information is used in a more or less 
straightforward manner in making a decision, and 
a conceptual mod i, in which "use" consists of 
thinking about the information. This article aimed 
to probe more deeply into the meaning of concep- 
tual use of evidence. The findings suggest that 
the conceptual use of evidence has three important 
features. First, conceptual use of evidence con- 
sists of something other than the accumulation of 
new information. Evidence is not merely attached 
to the user's store of knowledge; instead, concep- 
tual use is a formative process in which evidence 
is a "ted on by the user. Second, the formative 
process is not timebound; users may continue to 
form and reform their ideas as they confront new 
evidence, accumulate new experiences, or enter- 
tain new beliefs. The third important feature of 
conceptual use is that once users interpret evi- 
dence and draw inferences from it, it is these in- 
terpretations and inferences, rather than the evi- 
dence per se, that are incorporated into the user's 
store of knowledge and applied to working situa- 
tions. Often these interpretations and inferences 
are erroneous, and often they deviate consider- 
ably from the original evidence. Thus, when 
participants point to evidence as the stimulus of 
their decisions or their insights, what they really 
mean is that (a) their shared knowledge happened 
to evolve in such a way that (b) it could incoipo - 
rate the evidence, thus enabling (c) a particular 
decision or insight to appear to derive exclusively 
from the evidence. 

28. Kennedy, M. M. (1982), Working 
knowledge and other essavs . 
Cambridge, MA: The Huron Institute. 
(ERIC Document Reproduction Service 
No. ED 248 605) 

This document is the final report of a federally - 
sponsored research project on how school dis- 
tricts use evaluation and test data for managerial, 
instruci ; onal, or programmatic improvement. It is 
based on observations and interviews with school 
personnel in sixteen diverse school districts. The 
intent behind both observations and interviews 
was to expose the relationship between evidence 
- information from social science research, pro- 
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gram evaluation, test scores, and so on, and the 
working knowledge that school personnel had 
about substantive issues within their districts. 
Researchers documented everything that tran- 
spired during the meetings they observed, includ- 
ing any references to the use of evidence. 
Interviewers discussed issues of current interest 
to school personnel, with an eye toward docu- 
menting how and where different kinds of evi- 
dence fit into the interviewee's train of L\ ought, if 
it did at all. The results of this study are pre- 
sented in this report in a series of independent pa- 
pers. 

In the first paper, the author defines working 
knowledge as the organized body of knowledge 
that teachers, administrators, and policy makers 
use S[X)ntaneously and routinely in the context of 
their work; it includes beliefs, assumptions, inter- 
ests, and experiences as well as social science 
knowledge. This paper discusses the process of 
seeking out new evidence and attending to it, and 
the process of incorporating evidence into exist - 
ing working knowledge. Participants' descrip- 
tions of the process of seeking out new evidence 
suggest that the search process is active, contin- 
ual, and unsystematic. Evidence can be incorpo- 
rated into working knowledge in several ways; in 
each case, evidence is acted upon by working 
knowledge — sorted, sifted, and interpreted, and 
its original source and character are often lost. 

The second paper discusses how evidence is used 
conceptually by groups of educators, and how 
it is possible for group members to arrive at 
unified interpretations of evidence. The author 
found that groups of decision makers often could 
not use a piece of evidence in a decision until they 
had agreed upon its meaning, but in order to 
reach agreement, they often had to take 
considerable liberties with the interpretation of the 
evidence. 

The third paper examines the participatory politi- 
cal process of educational decision making and 
considers whether evidence had any independent 
impact on decision making. The author con- 
cludes that evidence did not appear to have an in- 
dependent effect. Instead, the participatory pro - 
cesses created a unified point of view that was so 
compelling that it imbued the evidence with mean- 
ing. 



In the fourth paper, the author examined the use 
of test scores as a management tool. She identi- 
fied four broad ways in which administrators 
might use test scores: consulting with teachers 
about them, using them to instigate competition 
among teachers, requiring that teachers respond 
to test score results in formal ways, and incorpo- 
rating test scores into teacher evaluations. The 
author found that these strategics and the stress 
they generated were counterproductive. 

The fifth paper looks at the role of in-housc eval- 
uator in these school districts, describes four 
types of evaluator roles, and argues that contex- 
tual constraints influence how evaluators do their 
jobs. 

29. King, J. A., & Pechman, Ellen ML 
(1984). Pinning a wave to the shore: 
Conceptualizing evaluation use in school 
systems. Educational Evaluation and 
Policy Analysis. £(3), 241-25). 

This article is based largely on the results of a 
year-long case study of a large city research and 
evaluation unit. It discusses the evolving evalua- 
tion use literature, the process of local evaluation 
use, categories of evaluation use, and the impli- 
cations of the case study results. 

Contrary to much that is written on the non-use of 
evaluations, there is evidence that evaluation re- 
sults are used by decisionmakers, but not in the 
immediate and direct ways anticipated. 
Furthermore, the results of research and evalua - 
tion units are not confined to end-products of 
■evaluations, such as reports, but also include the 
collection and distribution of much basic status 
information about the school district. The authors 
distinguish several types of use. The first is la- 
belled signalling, and refers to the use of evalua - 
tion information as signals from the local school 
district to funding and legislative support agencies 
that all is well, that legal obligations are being 
fulfilled and accountability expectations and re- 
quirements are being met. A second type of eval- 
uation use is called charged use. Like ions in a 
chemical equation, evaluation information thai 
takes on a charge, whether positive or negative, 
has the potential to cause a reaction in the system. 
It provides data upon which to base actions, and, 
to extend the metaphor, such use may occasion- 
ally cause sparks to fly. Charged use can be in - 
strumental, when an observable action is directly 
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linked to a specific piece of evaluation informa- 
tion and the user has considered the information 
prior to making the decision. Charged use can 
also be persuasive, when an individual selects 
information and puts it to use for his or her own 
ends. The persuasive action taken is not neces- 
sarily suggested by the information itself; the in- 
formation is instead sought or considered with a 
specific end in sight, then used accordingly. 
Non-use of evaluation results can also be con- 
sidered a form of use, either instrumental or per - 
suasive. 

The authors provide examples of each type of 
use, and of the process leading to signalling or 
charged use, from the case study research. They 
conclude with two observations: (1) an awareness 
of the specific evaluation context is essential to an 
understanding of the use process; and (2) what 
needs to be increased is not merely the use of 
evaluation products, but their appropriate use. 

30. Klausmeier, H. J. (1982). A re- 
search strategy for educational improve- 
ment Educational Researcher. 1 1 (2). 8- 
13. 

This article describes a research strategy devel- 
oped at the Wisconsin Research and Development 
Center for Individualized Schooling. The strategy 
has two main characteristics. First, it is directed 
specifically toward educational improvement; and 
second, it is planned and carried ou. jooperatively 
by a researcher or a research team and the 
participating staff of a local school. 

The author suggests three explanations for the 
lack of evidence that educational research has had 
an impact at the secondary school level. First, 
secondary school staffs may be unable to mount 
schoolwide improvement efforts, since the size of 
a secondary school faculty prevents the whole 
staff from meeting together, teachers' departmen- 
talized daily class schedules prevent smaller 
groups of teachers with mutual interests from 
meeting together during the school day, and there 
are few arrangements that enable teachers to share 
in the planning and decision making processes of 
the school. A second possible explanation is that 
educational research does not adequately take into 
account the uniqueness of each school, and a 
third possibility is that research has not been fo- 
cused enough on improvement in local school 
settings. 



The article then describes a project carried out co- 
operatively with five local schools and the 
Wisconsin R&D Center. The primary objective 
of the project rod of the participating schools was 
to increase student achievement in English, math- 
ematics, and reading by arranging an appropriate 
educational program for each individual student. 
To achieve this objective, student achievement 
was assessed every year and, based on the as- 
sessments, improvements were planned and car- 
ried out. Preliminary conclusions reported in this 
article suggest that substantial improvement in 
subject matter achievement typically has occurred 
when the school staff sets a specific improvement 
goal and then plans a:;d carries out improvement 
activities to achieve that goal. 

Finally, the author summarizes the four main 
characteristics of research that is focused directly 
on securing educational improvement in a local 
school. First, the research is planned jointly by 
an external researcher or research team and by a 
local school staff. The role of the researcher is 
that of a stimulator and consultant, not the direc - 
tor or the doer. Second, the school staff includes 
improvement-oriented research as part of their 
regular job responsibilities. Third, the school 
staff becomes increasingly autonomous in carry - 
ing out its improvement and research activities. 
And fourth, available knowledge is used in plan- 
ning and carrying out the improvement and re- 
lated research activities, and new knowledge is 
generated. 

(Note: The project is described more fully and fi- 
nal results are given in Klausmeier, H. J. 
(1985). Developing and Institutionalizing a Self- 
Improvement Capability: Structures and 
Strategies of Secondary Schools . New York: 
University Press of America.) 

31. LeMahieu, P. G, (1984). The ef- 
fects on achievement and instructional 
content of a program of student monitor- 
ing through frequent testing. 
Educational Evaluation and Policy 

Anabsis, 4(2), 175-187. 

Standardized tests have been identified as power- 
ful influences on teachers' selection of content to 
cover in instruction. A number of programmatic 
uf f orts have attempted to consciously link testing 
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with instruction; the results of frequent assess- 
ment with tests are used to inform instruction, 
with teachers tailoring their classroom practice to 
respond to the data which are generated. This 
study looked at whether a local assessment pro- 
gram, which involved setting twenty math objec- 
tives for each elementary grade level, testing stu- 
dents on each objective every six weeks, and 
giving feedback to teachers so they ^ould adjust 
instruction, resulted in higher achievement on 
standardized achievement tests (and not just on 
the locally-developed tests). 

Data from the California Achievement Test (CAT) 
were analyzed at the item level to determine those 
items that were substantively the same as the ma- 
terial assessed by the locally-developed instru- 
ments. Results showed that student progress on 
CAT items that overlapped with the objectives 
measured by the district tests was much greater 
than progress on items that did not overlap. The 
author concludes that the frequent assessment 
program is a powerful tool in enhancing the 
achievement of students. However, teachers may 
adopt the objectives of the monitoring program as 
ihc sole content of instruction, resulting in a rou- 
tinization of instruction and a loss of learning 
outside the content measured by the moritoring 
program. 

32. Low, D. Ability grouping: 
Decision-making at the secondary level. 
(1988, April), Paper presented at the 
annual meeting of the American 
Educational Research Association, New 
Orleans, LA. (ERIC Document 
Reproduction Service No. ED 297 023) 

This paper examines the processes used to form 
ability groups at the secondary level. The practice 
of ability grouping has been debated since its in- 
troduction in American public education in the late 
1860s. To date, the major portion of research on 
ability grouping has focused on its effects in four 
areas: academic outcomes, affective outcomes, 
socioeconomic and ethnic segregation, and 
classroom proceedings. Less research has been 
done on the formation of ability groups, and most 
of it has been at the elementary level. This study 
examines the procedures used by one school dis- 
trict to place junior high school students in ability 
grouped classes for English and science. Tnc 
study focuses on the decision-making process 
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and the placement criteria used by school person- 
nel to make their recommendations. 

In-depth interviews were conducted with nineteen 
junior high school staff members (teachers, coun- 
selors, and principals) from five schools. 
Participants were asked to make placement rec- 
ommendations for four hypothetical students. 
The results showed that staff members differed in 
the way they made placement decisions as well as 
in the result Placement recommendations for the 
four hypothetical students differed; no student re- 
ceived the same placement recommendation from 
all respondents. In general, participants shared a 
similar perspective regarding the differentiation in 
content, goals, and activities between the classes 
to which the hypothetical students could be as- 
signed. High lev_ classes emphasized higher or- 
der thinking skills and college preparatory knowl- 
edge, while low level classes emphasized lower 
order thinking skills and basic content. The par - 
ticipants differed in the way they considered three 
student characteristics - student interest, reading 
level, and home life - when making placement 
decisions. They also differed in their beliefs 
about the goals of the recommendation process. 
Some participants believed that the recommenda- 
tions should result in a placement where the stu- 
dent would experience the highest chance for suc- 
cess. Others believed that recommendations 
should result in placements where students would 
experience an academic challenge. 

These results have implications for policy mak- 
ers. Because school professionals differ in the 
way they make placement recommendations, the 
resulting placements could be functions of the 
teachers or counselors involved, not the students. 
Policy makers should consider articulating a spe- 
cific, preferred placement strategy that would 
provide direction in the interpretation and 
weighting of various factors. 

33. Morine-Dershimer, G. (1978-79). 
How teachers "see" their pupils. 
Educational Research Quarterly. 2(4), 
43-52. 

In studying the information processing of teach- 
ers, it is important to examine the concepts that 
teachers use to organize their observations of their 
pupils, for these are the concepts which form the 
basis of teachers' thinking, problem solving, and 
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der.sion making about pupils. In this study, ele- 
mentary school teachers were interviewed five 
times during the school year — at the end of the 
first day of school in September, shortly after 
teachers received pupils' diagnostic reading test 
scores in November, after reading lessons in 
January and March, and in late May, shortly be- 
fore the end of school. Teachers were asked to 
sort pupils into as many groups as they found 
meaningful, and then to describe the pupil charac- 
teristics they used for grouping; the process was 
repeated until teachers could generate no new 
groupings. 

Results indicated that teachers frequently sorted 
by six categories of characteristics: abil- 
ity/achievement, involvement in instruction, per - 
sonality, peer relationships, activity onentation 
(which activity the student had been working on 
that day), and growth/progress. The content of 
teacher conceptions of pupils shifted over time in 
ways consistent with changing teacher tasks. In 
September, when teachers were getting to know 
students, the focus was on pupil personality; in 
November, when classes were well into the in- 
structional program, teachers focused on pupil 
involvement in instruction. In June, looking back 
over the year and evaluating the results of their ef- 
forts, teachers focused on pupil growth/progress 
and on peer relationships, that is, on both the 
cognitive and social development of the pupils. 
At no time was pupil ability/achievement a domi- 
nant characteristic for teachers in their organiza- 
tion of information about pupils. 

Moreover, teachers tended to focus on different 
pupil characteristics in different observational set- 
tings. Pupil personality and peer relationships 
were important factors in their general observa- 
tions of pupils (in September, November, and 
May), while pupil ability and the activities in 
which they were engaged were important factors 
in teachers 1 observations of pupils in specific 
lessons. Different teachers also focused on dif- 
ferent characteristics, depending on their curricu- 
lum-management systems (e.g., whether instruc- 
tion was individualized). 

The findings suggest that teachers' organizational 
concepts are not rigid and unchanging. The 
teachers in this study exhibited an ability to use a 
range of concepts to organize their observations 
of pupils. Furthermore, the concepts used ap- 
peared to be responsive to the context in which 



observations occurred, especially taking into ac- 
count the time of year, observational setting, and 
curriculum-management system. 

34, Murphy, J,, & Hallinger, P. 
(1986), Transcript analysis: A tool for 
improving quality and equity in high 
school programs. The High School 
IflUinal, £8(2), 132-138. 

The authors of this article argue that one of the 
most powerful tools available for improving high 
schools is transcript analysis, the systematic re- 
view of student transcripts to answer questions 
about the quality and equality of students' educa- 
tional experiences. Although a few researchers 
have reported using this technique, transcript 
analysis is not being systematically employed in 
high schools today, either as an ongoing assess- 
ment measure or as an integral part of high school 
improvement programs. In this article, the au- 
thors review the technique of transcript analysis. 

Transcript analysis allows educators to monitor 
student progress by obtaining general assessment 
information, for example about student success in 
courses, as well as assessment information of 
critical importance at the secondary level, such as 
the selection of 1< ncal sequences of courses by 
students. Transcript analysis allows for the direct 
measurement of student opportunity to learn ma- 
terial and alignment of curriculum. It also as - 
sesses the equity of educational programs at the 
high school. 

A number of characteristics of transcripts make 
them useful in school improvement efforts. 
Transcripts are rich in information about courses 
and students; they provide a longitudinal perspec- 
tive that is both needed and often lacking in 
school improvement efforts; and they lend them- 
selves to use in conjunction with other types of 
information, such as standardized test scores. 
Transcripts also lend themselves to disaggrega- 
tion; that is, they facilitate the separation of 
important information into various groups; e.g., 
males vs. females taking mathematics courses, 
college preparatory vs. non-college prep students 
taking four- year sequences in English or science. 

The authors discuss how transcript analysis can 
be used to examine three aspects of coursework - 
total courses taken, academic vs. non-academic 
courses, and focus and sequence of courses. For 



16 



22 



example, in looking at the total number of courses 
taken by students, it is possible to examine the 
average number taken compared to units required 
for graduation, or the pattern of courses taken as 
students progress through grade levels. 
Transcript analysis can confirm or disconfirm 
whether a pattern of over-concentration in non- 
academic subjects is occurring in a given school. 
One of the most informative aspects of transcript 
analysis is the review of the logic inherent in the 
patterns of courses taken by students; sequences 
and patterns of courses can be examined to de- 
termine the extent of non-logical, non-sequential 
course taking. Finally, transcript analysis can be 
used to examine educational equity - whether dif- 
fered groups of students differ in the average 
number of courses taken, the concentration of 
academic courses, or the pattern and sequence of 
courses taken. 

The authors conclude with three points. First, 
because transcripts present information about stu- 
dents at a given school, the results of transcript 
analysis a:e site specific; solutions are not im- 
posed from above to solve problems that don't 
exist in the school. Second, transcript analysis is 
much easier if a computerized data base is used. 
Third, the power of the findings appears to be 
greater when the staff is involved in the process 
of analysis. 

(Note: Specific findings from a transcript analysis 
of five Illinois high schools are reported in 
Muiphy, J., Hull, T. R., & Walker, A. (1987). 
Academic drift and curriculum debris; Analysis of 
high school course-taking patterns and its impli - 
cations for local policy makers. Journal of 
Curriculum Studies . 12(4), 341-360.) 

35. O'Reilly, C. A. III. (1982). 
Variations in decision makers 1 use of in- 
formation sources: The impact of quality 
and accessibility of information. 
Academy of Management Journal. 2i(4), 
756-771. 

The direct relationship between the quality of in - 
formation used by a decision maker and decision 
making performance has been well established. 
However, the majority of studies of information 
and decision making have been done under con- 
trolled laboratory conditions that may not reflect 



adequately the circumstances under which deci- 
sion makers operate in actual organizations. In 
actual organizations, information may be contra- 
dictory or vague, available from sources of vary- 
ing credibility, and available at social as well as 
economic costs. Actual decision makers may be 
distracted, under time pressures, and pursuing 
multiple objectives. Moreover, a number of in- 
vestigations have documented differences among 
individuals in preferences for types of informa- 
tion, but little evidence is available that examines 
variations among decision makers in how and 
where information is obtained. 

The study reported in this article examines varia - 
tions in the reported frequency with which deci- 
sion makers use various sources of information. 
The rationale developed here to explain variations 
in use of information sources is predicated on 
perceptions of quality and accessibility of infor- 
mation and variations in individual and perceived 
task characteristics. The argument is made that, 
due to the ambiguity inherent in much information 
available to decision makers and to the pressures 
on decision makers to produce results, accessibil- 
ity of information may dominate quality as a de- 
terminant of preference for information sources. 

The study involved a survey of 163 county wel- 
fare workers whose dudes included the screening 
and processing of applicants. The participants 
were asked to respond to questions about the 
tasks they had to complete and the sources of in - 
formation they consulted while performing the 
different tasks. Results showed that when re- 
spondents perceived information to be of high 
quality, they also reported more frequent use. 
However, accessibility of information was a 
stronger predictor of use than was quality. There 
are several possible explanations for the domi- 
nance of accessibility over quality. First, deci- 
sion makers incur social and economic costs in 
seeking out quality information that is not readily 
available. Under time pressures, for instance, a 
more accessible source may be preferred to an in- 
accessible one, even if the quality of information 
provided is lower. Other possible explanations 
are that the structure of the organization may re- 
strict access to quality sources, or that organiza- 
tional incentive systems may have intended or un- 
intended effects on information search. 
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36. Schwenk, C. R. (1986). 
Information, cognitive biases, and com- 
mitment to a course of action. Academy 
of Managem ent Review, 1 U 2). 298-310. 

This paper examines the ways in which organiza- 
tional executives may promote commitment 
through the use of information. The author 
draws on literature in behavioral decision theory 
to explain the ways information may influence the 
decision processes of those who are asked to 
contribute to a course of action. Researchers in 
this area have identified a number of biases 
(which result from human beings' limitations in 
their ability to process information) and heuristics 
(simplifying strategies or "rules of thumb") which 
people commonly use to reduce the amount of 
information they must consider in decision mak- 
ing. 

The basic assumption underlying the model pre- 
sented in this article is that information may be 
manipulated by executives to encourage particular 
types of simplifying heuristics or biases by per- 
sons in order to increase their confidence in a 
course of action and their commitment to h . For 
example, research has shown that people's judg- 
ments tend to be biased by vivid anecdotal infor- 
mation, even if it contradicts more accurate statis- 
tical information. Therefore, executives may be 
able to influence persons' commitment by provid- 
ing highJy vivid information. The amount of in- 
formation available for decision making has been 
shown to increase individuals' confidence in their 
decisions; thus, executives might also promote 
commitment by providing large volumes of in- 
formation. Finally, since decision makers tend to 
seek out information which confirms their per- 
ceptions of control, executives can promote a per- 
ception that they are in control (i.e., that there is 
strong leadership) by providing certain types of 
information. 

The author discusses how commitment to a 
course of action can escalate, even in the face of 
negative feedback or failure. Also discussed 
aie conditions under which persons may become 
"entrapped" by a particular course of action. The 
article concludes with a brief discussion of ethical 
issues in information management and sugges- 
tions for future research. 



37. Shavelson, R. J., & Stern, P. 
(1981). Research on teachers' pedagogi- 
cal thoughts, judgments, decisions, and 
behavior. Review of Educational 
RfiSfiaidl, 51(4), 455-498. 

This article reviews research on teachers' peda- 
gogical thoughts, judgments, and decisions, 
drawing primarily on studies from 1976 to 1981. 
The article has six substantive sections: (1) a de - 
scription of the assumptions and methods of re- 
search on teachers' thoughts and decisions; (2) a 
discussion of the conceptual underpinnings of 
this research area and a review of research on the 
types of information teachers use to make judg- 
ments and decisions, the relation between charac- 
teristics of teachers and decisions, and teachers' 
evaluations of their teaching; (3) a review of the 
research on the cognitive processes underlying 
teachers' judgments and decisions; (4) a review 
of research on teachers' instructional planning; 
(5) a review of research on interactive judgments 
and decisions; and (6) a discussion of recommen- 
dations for further research. 

Among the many ideas and findings summarized 
in this article are the following: Research on 
teachers' thought processes assumes th^t teachers 
intend to be rational in how they make judgments 
and decisions, but that they must operate under 
the constraints that many teaching situations pre- 
clude reflective behavior and that human beings 
have limited ability to process information. Thus, 
teachers often seek ways to reduce their informa- 
tion processing load - by relying on routines of 
practice or by using heuristics, attributions, and 
other psychological mechanisms for simplifying 
decision making. Some of the antecedent condi- 
tions that affect teachers' decisions are informa- 
tion about students (their ability, social compe- 
tence, classroom behavior habits, etc.), the nature 
of the instructional task, and the classroom and 
school environment. Characteristics of teachers 
themselves which affect decision making include 
their conceptions of subject matter and how it 
should be taught, and their own cognitive style, 
including the ways they select and use inform a - 
tion. The research on teacher planning has 
shown that teachers are concerned with subject 
matter and content, primarily in order to build 
tasks. Teachers also consider information about 
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students, especially about their ability, when 
planning instruction. Specific goals and objec- 
tives do not appear to play a major role in the 
planning process, although they may be an im- 
plicit part of teachers' mental image of the plan- 
ning process. Since teachers' main concern dur- 
ing interactive teaching is to maintain the flow of 
activity, decision making during teaching usually 
arises when the teaching routine or activity is not 
going as planned, as evidenced by lack of student 
involvement or behavior problems. Typically, 
teachers choose not to change the routine in any 
major way, but incorporate this information into 
future plans. 

(Note: For a related article, see Shavelson, R. J. 
(1983). Review of research on teachers' peda- 
gogical judgments, plans, and decisions. The 
Elemental School Journal . 83(4). 392-413.) 

38, Sirotnik, K. A. (1984). An out- 
come-free conception of schooling: 
Implications for school-based inquiry 
and information systems. Educational 
Evaluation and Policy Analysis. £(3), 
227-239. 

This article discusses the use of a comprehensive 
information system for schools and districts that 
provides in-depth quantitative and qualitative de- 
scription of schooling and facilitates dialogue, 
judgment, decisionmaking, and action by those 
concerned with and/or responsible for schooling. 
The author refers to this general process as 
"contextual appraisal" or "systemic evaluation." 
The process is essentially formative and longitu- 
dinal, using feedback-revision loops for adapting 
to the changing circumstances of schooling. 
Monitoring student achievement progress is a 
fundamentally important part of the system, but 
"outcomes" are really pieces of the larger system 
and can easily be "inputs" when the system is 
viewed interactively and longitudinally. 

The author distinguishes between outcome-bound 
and outcome-free conceptualizations of school- 
ing. Outcome-bound models include input-output 
models, school effectiveness models, classroom 
learning models, and systems theory models. 
These approaches tend to rely heavily on mea- 
sures of student learning outcomes, usually 
achievement tests. Such models focus on the im- 
pact of student characteristics and school condi- 
tions in producing student outputs. The process 



of change in an outcome-bound model is fairly 
straightforward: conduct a needs assessment, 
identify the weak links in the system, infuse the 
system with improvements, and finally evaluate 
your efforts by looking for changes in outcome 
performance. In contrast, an outcome-free ap- 
proach looks at the school as a cultural ecology, 
where people are in a continual process of trying 
to make sense of, engage in, and/or adapt to 
structures and behaviors, in a mileau of feelings, 
attitudes, beliefs, and values. This conception 
suggests quite a different orientation regarding 
school improvement. 7 t suggests an inquiry ap- 
proach, where people actively and continuously 
engage in systematic deliberation about school 
improvement, trying to understand what goes on 
in their schools. 

District support, in spirit as well as substance, is 
crucial to school improvement, but the school is 
the primary focal point for bringing about im- 
provement and change. The implications for in- 
formation systems follow directly from this posi- 
tion. Top-down perceptions of the kinds of data 
relevant for schools are likely to miss the targets 
of need for school-based improvement. On the 
other hand, bottom-up perceptions of the kinds of 
data relevant for schools are likely to provide 
much information that is useful at the district level 
as well. Information systems should be opcra- 
tionalizcd with a multi-method approach to data 
collection, and should focus on analysis at multi- 
ple levels (individual, class, school, and district). 

39. Spady, W. G. (1988). Organizing 
for results: The basis of authentic re- 
structuring and reform. Educational 
Leadership. 4 6m. 4-8. 

In this article, the author argues that our educa- 
tional systems, schojls, and instructional pro- 
grams are not organized to ensure that all students 
learn well; instead, they arc organized primarily 
for student custody and administrative conve- 
nience. The prevalent practice paradigm in 
schools is both defined by and organized around 
the school calendar, and its adjuncts, the clock 
and the schedule. School decision making, cur- 
riculum planning, instructional and administrative 
operations, institutional arrangements, student 
certification and graduation systems, and student 
opportunity and eligibility conditions arc all dc - 
fined by and tied directly to the calendar. 
Carnegie units, credit hours, d <ss periods, grade 
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levels, and student eligibility criteria all reflect our 
time-based way of doing business. Even 
courses, which are commonly referred to as 
"bodies of knowledge," are actually time blocks 
that uniformly last nine months, no matter what 
the content to be mastered or the success of stu- 
dents in mastering it. 

In contrast to this approach, Outcome-Based 
Education (OBE) means organizing for results, 
basing what we do instructionally on the out- 
comes we want to achieve. Outcome-based prac- 
titioners start by determining the knowledge and 
competencies they want students to achieve, and 
then design curricula and instructional systems 
with the intent that all students will ultimately be 
able to achieve them successfully. For example, 
in the outcome-based paradigm, it is the out- 
comes, not the calendar, that determines credit 
and defines what corstitutes a "course." 

The author briefly describes an outcome-based 
education program implemented in Arizona and 
Illinois. The article concludes with a description 
of three key principles of OBE: clarity of focus on 
outcomes, expanded opportunity and instructional 
support, and high expectations for learning sue- 
cess. 

40. Sproull, L. S. (1981). Managing 
education programs: A micro-behavioral 
analysis. Human Organization . 40(2). 
113-122. 

Education managers operate in a complex infor- 
mation environment. Some theories of informa- 
tion processing and managing in organizations 
assume connectedness: information is connected 
to decisions; decisions are connected to action; 
action generates more information; and so on. 
Recent research suggests that the connections be- 
tween individuals and their information environ - 
mcnt may not be as stable or predictable as that. 
The concept of managerial attention is important 
in explaining connections between a manager and 
information. This article reports results from an 
observational study of five managers of major 
educational innovations who were nominated as 
being successful in their jobs. The study focuses 
on the structure and content of managerial atten- 
tion. 

The author sports five structural characteristics 
of managerial attention. First, attention is local: 



75% of the manager's day is spent at the program 
offices. Second, attention is choppy: the man- 
ager's workday is composed of many episodes of 
very brief duration. A third characteristic is the 
oral nature of managers' work: over three-fourths 
of the manager's workday is spent talking to 
people. The manager averages 44 separate meet- 
ings and conversations during the day. Fourth, 
manager's work is unpredictable; only about a 
third of the day i? spent in scheduled activities. 
And fifth, managerial attention is as much other- 
directed as self-directed: about half of the \ rba! 
contacts are initiated by the manager. In terms of 
the content of managerial attention, the research 
showed that the manager devotes relatively little 
attention to issues directly related to teaching and 
learning, only about 12% of the day. The rest of 
the manager's day is spent attending to logistics 
(22% of managerial time), external requirements 
(20%), and social pleasantries (11%). 

The article concludes with some observations 
about how these characteristics of managerial at- 
tention affect the relationship between the pro- 
gram and the larger bureaucracy, and the nature 
of instructional monitoring and feedback. For 
example, managers spend little time reading di- 
rectives from above, so that top-down models of 
educational change are rarely implemented exactly 
as directed. And the short, choppy rhythm of the 
manager's attention leads more to an informal, 
anecdotal assessment of teacher performance than 
to a systematic, in-depth analysis of the instruc- 
tional program. 

41. Sproull, L. S., & Zubrow, D. 
(1981). Performance information in 
school systems: Perspectives from orga- 
nization theory. Educational 
Administration Quarterly. 12(3), 61-79. 

This article explores the nature of administrative 
performance information in school systems. 
Administrative performance information is infor- 
mation collected by and for administrators about 
the performance of various components of their 
organization -- students, teachers, and administra- 
tors. The article is based on a study of 116 
school system central office administrators, in 
which the participants were surveyed or inter- 
viewed about their beliefs and practices concern- 
ing performance information. Results showed 
that every school system in the study adminis- 
tered standardized tests, but that test scores were 
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not very salient to administrators in their day to 
day decision making. Central office administra- 
tors attributed primary use of test data to others in 
the system; they believed that building level actors 
- teachers, counselors, and principals - find test 
scores :nore useful. Central office administrators 
preferred to use other forms of performance in- 
formation, such as personal observation, reports 
from teachers, parental and community feedback, 
and the achievements of students and graduates. 
They appeared to acquire much of their informa- 
tion in an ad hoc and haphazard way. 

The authors present a general specification of an 
administrative performance information system 
that can give both theorists and practitioners a 
broad and realistic framework for assessing per- 
formance. Such a system would be characterized 
by information collected from many sources, 
through many collection modes and several dif- 
ferent timeframes, and covering a wide band - 
width of data. The system would be accessible 
for use by any member of the organization. 
Information from an administrative performance 
information system would have a variety of uses, 
including providing a snapshot of organization 
status (scorecard use), highlighting a problem 
(attention-directing use), solving a problem, and 
demonstrating the organization's rationality and 
accountability (symbolic use). 

The article concludes with the observation that for 
school administrators, personal observation and 
conversation may be better sources of information 
for their day to day tasks than test scores are, but 
administrators may not be aware of how their in- 
formation collection strategies lead them to biased 
representations of organizational performance. 
Observation and conversation could be systemati- 
cally and explicitly incorporated into an adminis- 
trative performance information system, for ex- 
ample by calling teachers or parents at random to 
ask about how things are going, or by recording 
many details of an observation instead of only an 
overall impression. 

42. Stiggins, R. J., Conklin, N. F., & 
Bridgeford, N. J- (1986), Classroom 
assessment: A key to effective education. 
Educational Measurement: Issues and 
EUitta 5.(2), 5-17. 

Most scholars of educational measurement focus 
on the role of standardized tests in the assessment 



of student achievement. It is the perspective of 
these authors that that focus is too narrow; stan- 
dardized tests represent only a small fraction of 
the assessments that take place in schools, while 
little is known about the nature, role, or quality of 
the more common assessment measures devel- 
oped and used by teachers in the classroom. This 
article reviews what is known about classroom - 
based assessment. First, the authors review re- 
search on testing, which shows that classroom- 
assessments are often designed and constructed 
by teachers with little formal training in assess- 
ment. Teachers tend to use different assessment 
techniques for different purposes (e.g., diagno- 
sis, grouping, or grading); elementary teachers 
use assessment practices that differ from those 
used by secondary teachers; and teachers tend to 
assess students quickly. Analyses of teacher-de- 
veloped tests show a strong tendency to use 
short-answer questions that test knowledge of 
facts, rules, or principles. Teachers rely on their 
own assessments of students as the primary 
source of information on student achievement 
Little research has been conducted on the quality 
of teacher-developed assessments, but there are 
some indications of problems with quality. 

Next, the authors review evidence from research 
on teaching, which places the assessment process 
within the larger context of teacher decision 
making. When faced with planning decisions, 
teachers place greatest reliance on information 
about students' ability and academic performance. 
However, the focus of planning activity is not on 
student characteristics, nor on goals and objec- 
tives, but on instructional activity and content. 
Information on students' social behavior is used 
more during interactive decisions (while instruc- 
tion is occurring); however, teachers' use of so- 
cial cues to categorize students' intellectual com- 
petence sometimes results in misdiagnoses of 
student ability and achievement. Teachers use a 
variety of strategies to reduce their information 
overload and deal with the complexity of the as- 
sessment process, including sampling informa- 
tion from selected students, or assessing students 
on characteristics that are easily measured, such 
as recall of facts. 

The article concludes with a discussion of impli- 
cations of these findings for classrooms and 
schools, for further research, and for training. 
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43. Williams, R. C, & Bank, A. 
(1984). Assessing instructional 
information systems in two districts: The 
search for impact. Educational 
Evaluation and Policy Analysis. 6.(3), 
267-282. 

Despite differing approaches to decision making, 
scholars generally agree that information can and 
should play a vital role in effective decision mak- 
ing. Organizational decisionmakers can be 
thought of as having three kinds of information 
sources: working knowledge (information gath- 
ered from informal or experiential sources), des- 
ignated information requests (such as asking a 
trusted source or commissioning a specific 
study), and designed information systems (a for- 
mal, continuing process of collecting information, 
such as with a management information system). 
Public school officials have been criticized for 
relying too heavily on working knowledge. One 
answer to this criticism has been the development 
of educational evaluation. Thus, beginning with 
federally funded projects in the 1960s, foimal 
evaluations were intended to provide reliable, 
comprehensive, and unbiased information about 
the efficacy of particular programs. An emerging 
movement in education is toward developing dis- 
trict-wide instructional information systems that 
can provide a variety of users with information 
that will help them make instructional^ relevant 
decisions. These systems can potentially gather 
and link together data on factors such as pupil 
characteristics (ethnicity, language skills, mea- 
sured academic aptitude), pupil performance on 
tests, instructional methods, and attitudes and 
opinions, to inform decision makers. 

This article examines the use of such information 
systems in two school districts. The researchers 
examined, through interviews with school staff, 
the expressed satisfactions of individuals and 
groups with the information they received from 
the instructional information systems, the pro- 
cesses they engaged in, and their self-described 
behavior in relation to the information and how it 
was used. The researchers also examined unan- 
ticipated or unintended effects of the information 
systems. 

In the first school district, the instructional infor- 
mation system contained information from needs 
assessments of parents, educators, and students, 
along with standardized test score data. The sys- 



tem was designed to provide local school site 
councils with information for use in three stages 
of decisionmaking; identifying and clarifying 
problems, generating and selecting solutions, and 
monitoring solution effectiveness. The re- 
searchers found that the information system 
played only a minor role in decisionmaking by the 
school councils, although information from the 
system did sometimes corroborate perceptions 
that were initiated through other means. This 
may have been due in part to the fact that the in- 
formation maintained through the information 
system was limited in its scope and accuracy. 

In the second district, the instructional informa- 
tion svstem was designed to help teachers contin- 
ually tailor their instructional practices to the 
learning outcomes of their students. The system 
consisted of a series of criterion-referenced tests, 
measuring student progress on objectives in a K- 
8 curriculum for reading, language arts, and 
math. The tests are given to students at least three 
times a year, and scores are reported quickly for 
individual students and also by class and by 
school. Test score results are used to plan a staff 
development program, available to all teachers, 
which provides training in the diagnostic pre- 
scriptive teaching skills which are felt to be nec- 
essary to act upon the test score results. Each 
school also has a learning specialist, who func- 
tions as a resource to teachers and also provides 
extra instructional time for students who need it. 
The researchers found that the information system 
in this district was used often to influence teach- 
ing practices and the planning and monitoring of 
instruction. Teachers were articulate about the 
positive uses of the test score-based system, al- 
though they expressed some reservations about 
relying exclusively on test information on stu- 
dents. The system appeared to have several rip- 
ple effects; it facilitated communication among 
teachers, principals, and parents, and it was as- 
sociated with a sense of collegiality and shared 
values in the schools. 

The researchers hypothesized that the different 
levels of use of the two instructional information 
systems depended in part on the significance of 
the decisions which the systems were meant to in- 
form, the compatibility of the system with the cul- 
ture of the schools, and users' perceptions of the 
validity of the data. 
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